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The primary objectives of ^^hi,s project commonly 
refere'ed to as "Assessing, Human AbiJ-ities" wer.e: f1) to provide 
"^^c^^fexence measures for cognitive factors; and (2) to* provide a guide 
to\re|erence measures for self\^eport temperament factors. The^ . • 
overall objective was to conduct research in the area ^of factor 
analysis directed, toward fUe identification of tests and^ dther 
instruments that can serve as markers for well-established factor's.' 
The term "factor referenced" was iiitroduced in this project to call ^ 
attention to 'the factors as the construct of interest. This usage is.* 
in keeping with the general conc^t of crit4rion^ref erenced tests. It 
is the^ sense of ^c'larif yingVsomething that is beilig measured that *^the 
f actor-refei:eneed nieasures jare offered* to researchers and 
practitiocers concerned Hith .assessing 
thd!^ th'e. results of this; studj 
structure for .the cognitive doma^; 



°foiLa cq/mparable structure for w ^*w«v*^-. ^ ^ 

Such^ theoretical structures ai:^ founded on empirickL' evidence and are 
amena^bl'e to' continued challenge and verification. Besearchers could 
be^expected to use % small number of the resulting f actor^ref erenced 
tests as markers for testing conjectures about , factors in ^heir 
^studies. Summary lists q^I the 23 cognitive fjactors with the 
recpBimended marker testsT^and the 28 temperament factors with the 
scales .proposed for 'them >ar^ given in^' ttfe appendix. (fiC) . ' 



human abilities. It is hoped 
in the development of , 
uman abilities and a begilining 
rament domain of personality. 
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PREFACE 

■ ' \ " ' •' J 

Psychologists and educators have long sought to identify 'dimensions of 

aptitude and ability* and have also been concer^ned witK sici^lar qijestions^ 

in 'the personality domain. Conceptually, the investigator would like to 

think of all' possible tests that currently exist or might bfe constructed in 

the future, say in the cognitive ai^ea, so that a factor analysis would yield 

the "factors of t>e^mind." Then the/ tests that best measured these, f^^ors 



would be desigxiated as tfejA reference tests for >the factors. This type of 
ab St taction is the behavioral scientist's counterpart <q>f the chemist's. 

periodic^ table, or the physicist's '^'standara) meter. " 

[ 

An alternative approach to tW identification of the factors of Is^raian 
ability ts through the synthesis of the myriad research effortSL^ many 
individuals over time. This consists of a continuing process including: 
^^me^culpus search and refinement of measures of abilities; review and 
attempts at matching^ and consolidating them into .meaningful categories; , 
empirical checks of results; and itera^ioA of this process for clarification 
and improvement. It was in the spirit of cjntributing to this approach tftat 
the present project was undertaken. Hopefully, this study will be of help, 
at least in some small way, to the many workers striving to^develop a 
conceptual framework for assessing human ^bi.lity and tenqperament. 

^Our study, aimed at identifying tests and other instruments to, serve 
as markers for factorlp of , huiAan abilities, has sponsored tiy trie Office 

of Naval Research, Psychological Sciences Division, personnel Training 
Research Programs. This support is gratefully acknowledged. In particular, 
we wish to express our appreciation to Marshall J. Farr, Director ,v and 
Joseph Young o^the Personnel Training ReseaUch Programs for their assistance 
in facilitating the research effort (as well as Victor Fields, who served in 
. . i . ' 




this ciipacity during the first year of the project). We are also grateful ^ 

■ ' . . * O 

to the NaVy Personnel and Training Research Laboratory, San Diego, for , 
enabling us to do the field experimentation. Bernard Rimland, Department"^' 
Director, Edmund D. Thomas, and James Stapleton Were most helpful, not only 
in expediting the test admlnlstratib^ 6ut al^ by making contributions to 
]]the substantive aspects of this study. ^ 

The followdiig panel of advisors was constituted early in the project: 



Peter M. Bentler 
University of California, 
Los Angeles 

Raymond B. Cattell 
University of Illinois 

Lewis R. Goldberg 
University of Oregon 

J. P. Guilford 
Unlve^rsity of Southern 
California 

Chester W.^ Harris 
University of California, 
Santa Barbara 

Margaret L. Harris 
University of California, 
Sanfea Barbara 

Douglas IT. Jackson , 
University of Wesrtern 
Ontario 

Maurice Lbrr 
Catholic University. 



Philip R» Jlerrifield 
New York University 

John R. Nesselroade 
The Pennsylvania Sjtate 
Ujiiversity 

Warren T., Normal 
University of Michigan 

Bernard Rimland 
Naval Persdnnel and Training 
Research Laboratory 

Saul B. Se^s 

Texas Christian University 

Calvin W. Taylor ^ 
University of Ut^ 

/ . 

. Ledyard R TUcker 
University qf Illinois 

•Jerty S A" Wiggins • 
- University of Illinois 



r 



As a grotJ^p, they had considerable influence on the direction of the project 
and we are grateful for tWir guidance". Furthermore, the project 'staff 
benefited greatly from the advice' and counsel offered by individual members 
in the final stages^f the project.- ' ■ \ 

A - • • ii ^ 
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.Other advisors who^e "help we acknowledge .are the* following members of 
the ETS staff: ' • ' ^ 



John B. Cairroll Richard Levjlae (d&eased) 

(now, Unirer^ity of North 
Carolina at Chapel Hill) frederic M. Lord 

Norman Frederiksen \ Sdinu^l Messick - , " 

Harold Gu^liksen , Lawrence J. Strieker 

The project staff rec^ved assistance in a variety of areas from visitors 
and regular staff at ETS, the most important of whom were: ^ 

Edgar Howarth, University of Alberta, for review of noncognitive 
factor descriptions from draft -copy of Guide, while spending part 
of 'hia sabbatical leave at ETS (1975)^ 
Joseph R. Royce, University of Alberta,/ for st|mulating discussions- 
.on the psychological insights offered by the factor approach to 
the development of substantive psychological theory, during ^is . 
' visit at ETS X1973>^ 

W. C. E. Young, University of Nairobi, for review of personally 
items when he sp'kv£^art of his sabbatical leave at EJS (1972). 
John L. Barone, ov^all guidance in computer programming. C 

^Johri J. Ferris, experimental * design. ^ 

i . ■ ■ ^ " " 

kenrietta L. Gallagher, directing scoring of divergent productioi^ 

. , \ - u ■ 

tests. 

lirene Kostin, initial literature search in the . temperament ^rea. 
Gita Wilder, writing many of the personality items. 
Brknda J. . Brown, %athe^ine Farley , Sharron Fouratt, and C- Brooke - 
iruenberg, typing assistance for all phases of the study. ^ 
To all of theipe people y our sincere thanks. 

^ - iii 
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, FINAL REPORT 

.■ . ■ . . ' 

.t^^ IffiSEARCH ON ASSESSING HUMAN ABILITIES 

.... ' ■ <J 

; ' ■ ■ • \ , • _ 

Overview ; - ^' 

Tfhis is the fifial report on a- research study sponsored by the Officer of 
\ * ' ~ . di -• 

Naval Research uriider Contract N0J3014-71-C-011 7, NR 150 329. Al^tiough the 

project acquired the short title,; "As^esging^uman Abilities,'/ its-^cus was 

explicitly oh two areas: (1) to pr|>vide reference measures for cognitive 

factors; anil (2) to provide a ^ide ti) reference measures for self -report 

temperament factors. The overall objective of , the project has been to 

coaduct research in tW area of factoV analysis directed toward the identifi- 

cati^on 9f tests aud other instruments that can serve as markers for well^^ 

established factors. ' The total effort of the fout-year project miy be viewed 

In Retrospect, as consisting of three phases:" Planning (approj^^ one, 

yeai), executing plans (roughly, middle two years) , and deveOoplng final 

products "O-ast year, plus some slippage). * . ^ 

' * 

During the planning phase, consideration wa^ given to problems of up- 
dating the matker tests for cognitive factors and iStlSit directions and means 
might be followed for developing marker measures for the personality domain. 



An important part in our planning was the convening of a .Conference o 
March 29-30, 1971, for the purpose of getting ja current reading of^he 
status of reference measures for cognitive and noncognitive factors. The 

advisory group of specialists in the area of factor analysis and Jiiuman 

■ . ' * ' \ 

assessment was chosen to represent the professionals who would be most likely 



tcif»se such re'ferenbe measures in the future. The Conference- was designed 
to give an opportunity *for the advisors to help us develop, plans that would 

• -1- ■ 



assure Vhe maximum utility of the resulting reference measures to the pro- 

fession. 'It was, also our hope that the participants themselves would gain 

* . • . * - . • 

something from one another • Those attending |the Conference included the 

panel of 16 advisors/ seven ETS advisors, and a representative of the Office 

of Naval Research, as well as members of the project staff. 

O Some, general guidelines that emerged Vrom this Conference were the. 

foilowingtV; ^ <^ ' ' ^ • "^"^ ^ ^ 

(1) , A factor, will be considered as "est4blished" and markers for it 

will be provided as end'products of this project if it is possible 

to identrfy^it in at' least three analyses performed in at least 

two different labbratoriea-. - ^ ' i 

f - / ■ . ' 

(2) At ;j.east three test^'will provided as mai^kers for each 
■ ^ • ■ ■ ' :^ , ^ ■ ' ' \' ■ ■ . ■ . ^ \^ . ' ^ 

estal^ished cogiiiti\|B factor; at leal^l fouf measures will be' 

• . / > A " ' \ 

provided as jnarkeirs for an established noncognitive tactor, two 

' * I' % ' i ■ ' ^ \ * • 

for each of ' the opposite poles. . . - 

• ' I .» . V. . > . ' 

(3) Newly developed testei and other measur.es for both the cognUl^ve ^ 
andTnoncognitive domains will be field tested in order to determine 
some of their bafeic statistical properties (e.g.i reliabilities and 
item difficulties) and to check their factorial content. 

Thus, the objectives set f^ this study inpluded updated reference tests for 

cognitive factors and at least a befi|miing of reference -measures for non- 

s , . ' 

cognitive factors. ^ 

* • » ^ • 

. Toward the end ofi the first year of the study, and for the next two 

years i our endeavors followed two principal lin^s: ,1) a thorough search of 

the literature for established factors; and 2)^ verification through field . 

testing, including the development of new peasuiofes^S necessary. 



Th^ final phase. of the project involved the consolidation <rf the wpiri- 
cal results with the more general professio^l findings to provide reference 
measures for cognitive and noncogniti^e factors. The end prpducts are a , 

revised Kit of factor-referenced tests for cognitive factors and a Guide to 

«... ' • 

factoid-referenced scales for temp6ram^nt^ factors. • 

. ^ In this project we introduced the*term "factor-ref eren(?Sd'' measure to 
call attention to the factor as the construct of interest. This usage is. in 
keeping with the general concept of criterion-referenced tests. In a jphilo- 

i * ' * V '1 

sophical paper on-questions of meaning of psychological meV^ureinent, Messjjj^k 

' . . . ' • fel - . , ) ^ 

, considers the topic, "referencing measures to interpretation and us^;" in 

which h^larlffies the oft-found fuzzinesS in the use o^^Vm-r^ferenced and 

criterion-referenced measurements. He calls attention to .th^* wide acceptance 

of construct validity as "the touchstone of interpretation and meaning in . 

psydtiolofeical measurement," and stresses that " . > - all measurement should 

be construct-ref erenced . A measure estimates how much of something an indivi- 

^ ' [ ^ . " * <\ 

dual displays or possesses. The basic question is, 'What is the nature of- 

' ' ' ■ • A . _ 

tlj^t something? ' It may be answered by referring oto evidence in sj^pport o*f ^ 

■ '■ ^ ■ / ' 

particular attributes, processes^, or" traits con^rued to underlie and deter- 
mine task performance." " % ^. 

rt is in this sense — of clarifying that.^something that is being 
measured — that the factor-referenced measures are" of f ered to jrpsearchers 
and practitioners concerned witl^ Assessing .human abilities. * . 



^ Messick, the standard problem; Meaning and values in measurement 
and evaluation. Research, Bulletin 74-44. Princet.on, fl^ J. r-^^^ducitional 
Testing Service, 1974. (Will appear soon in The American Psycholqgis t . ) 



* 0 - . 

, The aim of 'this study was to provide a reference basis for diffej'ent 
researcheij^^ in their' combined efforts to conceptual j^ze and deVelop a .theory * 
and structure of human abilities. ,It is our hope that the results of this 

. • ■ - )'• - ■ - ^ • 

.rteseajc'ch will aid in -^e de^velopment of a structure* for the cognitive domain |^ 
•of ^hiuxian abilities^'and a beginning for a comparable structure 'f&r the tempera- 
ment domain of personality." Such theoretical structures are founded on . * 
eiotpirical evidence antf are disenable to continued challenge and verification. 
Researchers could be expected tp use a suiall nuniber of the resulti^ng^ factor- 
referenced tests'^ as piarkers f^Dr testing conjectures about "factors in their 
stuaies.^ - - . ■ 



• ^ A natural « appi^oach for tl\e development of such a bas^s is to build upon 
the collective prior efforts. Qf course, the 1963 revised Kit of reference 



^teats fpr cognitive- factors g'avfe us d point of departure. The factor analysis 
!(^nference held in- the Spring of 1971' served to provide the current '/state q& 
the aft"'wlth respect to factor-analytically-derived measures of cognitive ^ 
dnd nojicpgiiitive trait;,s. More concretely, the extent of empirical evidence 

f or 'well-estabMshed factorls could only be determined by means of a thorough 

' ^ .> • , * 

literatu^ search. " For factors so determined there arose the creative task 

-% ' * ^ 0 " " • ' ^ ' 

of preparing -suitable items as markers for the^m. Then, when a sufficient 
number of items had been prepared^ they "were "packaged^" into tests or scales 
.for field administration. TSuch pretesting or field tryouts served to deter- 
mine test xell^bilijiy, some normative information, and some verification of 

.the effectiveness of 'the. inarker fce^ts in id^tifying the postulated factors. 

' • " ^ • ' / ' ■ * ' • . , . 

The foregoing approach has -led to a much improved reference iCit for cognitive 

^ ' \ ■ ^ . * . 

factors and to a good start for a, ^reference Quide for noncognitive' factors, 

i> * * • * 

self-report temperasient measures in particulai;. . ^ 



/ The literature search indicatied the present status of the* 24 cognitive 
factors listed 'in the 19j63 Kit^and some half -dfjzen potentmLly new factors 
'that' have appeared since sthenl Iii the noncoguitive a^ea, ^ total 'of 28 
factors in, the temperament domain were, identified' thr9ugh thel literature 

search^ These findings are summarized in the fol3.owing reports: 

■ \\ ° " ' < 

TR 1. Toward the establishment of noncoguitive factors ' .^..o 

^ through ll^terature search and interpretation 

" (John W. French) . " 

TR -2^ Cognitive factors: Som^^ecent literature^ 

* (R^th B. Ekstrom)^ • , ' v 

/ The work of updating, modifyinLg, and extending the 1963 Kit of cognitive 

tests involved the following thr,ee activities:^ 

(1) ' Development of new' divergent production t^asks; 

(2) Review and modification of other tests of the 

. 1963 Kit to 1>e included in the jiew^t; . . 

(3) . Development of , new marker tests for factors 

that have been ^tablished in; the^literature . 
since 1963. . * • 

After the modification and development of the n^w tests, the empirical veri- 
fica^ion of their usefulness in marking the putative 'Actors was carried out 
rby m^ans of two field experiments ^ ^ 

The first field tryout's covered 23* tests designed as markers for^seven 
hypothesized cognitive factors related to divergeOjt production. Sinqje it 
was not feasible to give the total battery of tests *A:o all subjects > our 
experimental design was such that no man had to take more than seven of these 
tests, while still providing sufficiently large N*s (average 187) for 



ealciilajtian of reliabilities and some factor analyses af subsets 
-Ten different subsets wer^^onstituted ^Ln order to investigate the .identlfi- 
'ability a^id. degree of independency of the following factors:. (1) issociational 
fluency, (2) ^3cpressional fluency, (3) originaliCy, <4) semantic red^finitlon^^ 
(5) sensitivity to problefiis„ t^) figural' flexibility^ and \7) seimantic fiexi- * 
bilityw The results are Resented in the following report. . ^ / • 

^ TR 5. Problems of replication of seven divergent 
productidii factors (RuClh EkstroDgi, 
John'H. French, and Harry H, Harman). . 
In subsequent field work t-ze improv^JTour experimental design/ employing 
iliatrlx sampling/>to enable iis to calculafc6 .correlations arol^ all variables 
and thus do llactor analyses involving \all o!E them. Such a design was used in 
another field tryout of 33 cognitive tests f^Ir 11 different^aJtors. The aim 
of this part of the study wap to investigate the five nei^ly established factors 
from the literature, namely, concept |ifctainment, figural fluency, integrative 
processes, visual mempry, and verbal closure. Because lit^ seemed especially 



important to determine whether tht new factors could be separated' from other' 
somewhat' similar fact^^, marker tests for six of these older factors were 

inclu3ed lln the study. These factors are figural adaptive flexibility, 

» *. 

logical reasoning, general reasoning, number, spatial orientation, and speed 

of closure. ^ y 

The new desigiNinvolved the adaptation of a 21 x 29 ^irmal statistical 

experimental design. The tests for the 11 putative factors x^ere grouped 

into 21 "elements" in an attempt to reduce the variance in testing time of 

♦ 

the 29 sessions. Each of these "elements" (consisting of one, two, or three 
tests) "was a^^ninlstered in six different testing sessions*^ That' is how % 

^ U ' ' 

14 " 



large N's were obtained f6r overall 'item ''and te§t statistics. However, for' 
a cancelation between two sets/ the N was :0jalted to the individuals taking 
the pair , of tests in a given session/ ^Further details of this design are - 
included' i^ the following report on tHe Ixex^r cognitive factors': ^ 
'V ^ , TR 8. An attempt to ^cKjnf ixm. £iv§<^ recently identified \ . . 
- : ^- . r V . ^fif^gtiSrive; factors (lRuth B.^Ekstrom, John W. ,) 

*■ ■ • *" • ■ 

^ '\ ' ' French, and Harry ji. Hatman) ."^ 

M addit investigation of cognitive abilities is covered in the 

f oll(w^^\heoretical (^aper : 

/ TR 4. Psychometric tests as cognitsLve tasks: A new 
«^ "stnlcture of intellect" (John B. Carroll). 

From a detailed subjective analysis of the cognitive processes involved in 

\ . ^ 

the tests of the 1963 Kit, the conclusion is dra^pi that cognitive taslcs are 
complex, and cognitive factors resist classification by any rigid taxonomy 
such as Guilford's Structure of Intellect model; there, are probably no such 
things as truly ".pure" factors. Thus, a study of individual differences in 
abilities can profit greatly if it Is closely tied to the experimental 
^nalysis of particular cognitive tasks. 

Research in the noncognitive area proceeded ih parallel v;ith that in ^ 

the cognitive area, although -v/e had a head start in thej latter. In the ! 

^ ' ^ ^ / 

noncognitive area it was possible to complete the literature search' prior 

if 

to any field testing. Some of this, work had already been done *at ETS prior 
to the start of^he project in conjunction with other research interests. 
Pertinent data wer6 recorded for all studies in which a temperament factor 
was found. X^en a factor was found in three or more studies emanating from 
at least two different laboratories, it was retained^as an "established" 
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■ ■ ■■ \ • - ' 

• * " ' • * 
faetor/ This requl^red, && noted in Teclmical Report^No. 1, a great deal 'of 

.''•-subjective judgment regarding corresponding factors, for it was rare^^for 
different inv.estigators to do andlydes involving conHnon sets -of variables. 
/A total of 28 ^uch factors halre' been identified/^ ^ . ► 

, ♦ For each of the honcognitive factors, we tried to^ include as markers 

• the different concepts, or subscales, associated with them in jthe y-teratuire. 
Items were writteif for each* subsdale .taking into t^accouijit. the following needs: 

' ' ^ ' ' ■ * . V ' V ' ' ' * • 

(a> bali^cing the well-known diffefrences-* in tending to ^c<Iuiesoe tb a self- 
report questioimalre item, and (b) considering both of the opposite 
psycholo'gical poles that seem to te i^art of most temperament trjaits. A 
scheme that we followed for meeting these requirements involves four ited& 
for each concept so that a person "high" on a factor (i.a*^ direction iti 
whiQh the factor is named) would be expected to respond as follows: 
f'^- '^r^' '^'^^^^ - "yes" on an item designed to be positive on ' . ^ 
^ W -'-^^-i^'"''^-^ the positive pole of the factor; < 

\> . ^ , f , • 

* (2) "no" on an fitem designed to be negative on 

■ ■> \ 

' the positive pole; 



(3) "no" on ah item designed to be positive on 



the negative polej ^ * 

(4) "yes" to an ifem designed to be* negative on / 
the negative pole,, 

Another way of stating this is by way of keying four such items intended as 

/ * * * 

measures of a factdr^ namely: 





Yes 


No • 


statement 


Pole 


(1) 


1- 


0 


Positive 


. Positive 


(2)- 


0 


,1 


Negative 


Positive 


(3) 


0 


1 


Positive 


Negative 


(4) 


1 


0 ' 


Negative 


Negative 
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This plan led. to four se^tsVof such four iteriis each, prepared by three 
-independent workers to avoid bias and in o'^der -'to iijcrease the reliability^ ^ 
of meksurihg each factor. Furthermore, sitice each factoiT is "exemplified by V 
:seyeral different conceptp,.^wlth a!| average of about three, subscales pei;, 
.factor, the aggregate' numbe^ of items^ to coT^er the 28 factors jiame close to 

1400. ' ^ ^ \ • ' ' . ' \ : 

•After all these items wer^developed it iwast^nec^ssary to put them in 



some reasonable "and practical fotm for field ^tryout. Whatt^cpnstitutes a 
noncognitive instrument is nott as easily formMatfed as what constitutes a 
cognitive test, in the case of an abi^lity tes^t, a, c6mmon procedure is to 
select ipem^ that presifinably n^ka^ure the desired ability and arrange them 
iLn^^or^r|of difficulty, inclu^iiig an appropriat^e number to keep the most ^ 
able* of t^^ntende'd'lopulaitloii occupied for the planned time period of the 
test. For noncogrfitive mea,sui?es it would not do to present, in sequence, ^ 
all items intended to measure^ a partlpula^r trait. The several' versions of 
y particular item must be sjpread throughout the instrument.^or else the 
examinee could infer the tiait sought ayid \respond as he x-zished to be 
perceived rather than aS he is. The ifeenisl intended to mark given factors 
and subf actors were approprjiately coc ed so that they could be 'identified, 
sorted, and scored accorddnig to a ^ predetermined key. ^ ^ > > 

> to^a 'field test of the 28 factjbr-referenced scales, the items x^ere 
put fii\*30 booklets (with repetition^) so that they could be administered 
in as ri^any sessions to more than 4000 men at the Naval Training Center^ | 
San Die^o. By means of a rather complex design, it was possible to limit 




g time for any individual 
enabling/phe determination of .relal 



to no more than 320 items, while still 
ionships among all 87 subscales designed 



i7 



to mark thi 28 factors. A report,^ entitled/^^perlmental design for a study . 
^ # * i " • ' 

• of self-repbrt personality items," was presented by Harry Harmon at a 

' meeting of the Society of Multivatiate ExperSmential Psycholo^in November 

1972, at.Foi^t Worth,* Texas. As a result of this r(Bport, Dr. Lewi^R/ Goldberg 

.volunteered to administer the full sejt of 1400' ijtems to a sample of female 

* and male students at th^/<Universifey of 6reg9n. „ 

^ Thejfirst results ^stemming from the field tests in„^nl)iego were 

reported at a symposium presented J)ef ore the^American PsychcJlogical Association 

" / - V * • '^^^ . • " . . . . 

in August 1973, in Montreal » Canada, ^ It was our work in the- p^e^rsonality •area 

■ . .r . ' - - J* ' ' ' « • ^ . • ' 

^ . ^ h ' ■ ■ . ' - . ' 

that -served as a point; of depaftutre foi^ planning the'' 9ymp9Sium wh^ch has since 

■ •^<f', *• . 

^ b^en publiahjB^ in the following technical report: 

♦> \ . - 

' ' TR 3. Proceedings: -Toward the development of more 

. ■ • " . - \ ' 

< . comprehensive sets of personality measures' . ^ 

. " V ' " " . / K • .,/\. " . 

^ (Harry H.;^rman, Editor). ' . . " 

The final results, covering the college sample as well as the Navy sample, are 
presented in the following reports; 

TR 6. Verification .of s^lf-report temper^ent factors 

(Diran Dermeh7 John W. French, and HaiTry H. Harman) . 
TR 7. Seeking markers'^ for temperament factors among 
^ ^ positive and negative poles ci)f temperament 

scales (John W. French, and Diran Dermen) . 
Before the last report was published the material was presented by" John W. 
French at a meeting of the Society of Multivariate Experimental Psychology 

in November 1974, at London, Canada, and the res'iilts. of the discussion were 

• * ' \ 

incorporated in the report. 

• • • 
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Final Products 

In this research study the output inc^ludes not only the eight technical 
reports covered in the last section, but aJ^so several end products. These 
include the actual Kit of faetor-referenced pognitiv^ tests, a Manual for the * ^ 
use of these tests, and a Guide to factor-referenced temperament scales. 

While the stated objective of the study was to develop two kits of tests 

— one for the 'established cognitive factors and another for noncognitive 

factors — the solution w^s not as siiqple as' that. The Kit of reference test^ ^ 

for cognitive factors involved, in large measure, the revision and updating of 

material that had been under development for more than 20 years, with the last 

published version in^l963. Of course, there had to be introduced some new 

* "established" factors, and tests to measure them had , to be developed. Hence, 

the new publication is very' similar in form to the preceding one but has been 

f|iiiproved to provide more ready accessibility as well as the .inclusion of the 

latest factors found in the literature and substantiated in field tests. 

*Since the development of reference mateprials for the noncognitive domain 

♦ 

was a new endeavor for ETS, it entailed many special problems. First, the 
area of concern had to be delimited. The personality measures under 
consideration are limited to the area of temperament for normal adults. 'More 
specific limitations are spelled out in TR 1. After defining the area of * 
concern, there still remains the question of how to provide reference materials 
for such factors. There is nD ready counterpart to cognitive reference tests 
that can be easily reproduced for research purposes. Even ready-made 
personality scales usually cannot be used directly but must be interspersed 
with other scaled. More importantly, it wot^ld have been very presumptous ^f 
MS tQ assume thit we could produce independent scales for all the personality 
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factors indicated in* the literature. Therefore, in this first issue of 
personality referenced materialist .our thesis has been to take thQse factors 



that have been "established" according to our criteria and to .refer to 
existing -scales, indicating the extent tb which our empirical efforts 
Support the {)artidular factor. Where we had substantial success, we include 
.our new scales. Thus, for 26 of the 28 nonc^nitive factors, jreferenceS are 
given oti^^Cales develo|fed by other researchers and scales developed^rln, the 
present j:esearch' if they held up in the experimental studies.- v."" - *. 



It is ^of interest to note the progress 



made over the last 20 or more 
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years. The first Kit (1953)- of reference te^sts f or ^tHe better established • 
cognitive fact^s included 16 factors with only specime;i reference ^ests for 
; each of them. None of these tests was developed at ETS. In subsequent years 
a number of additional factors and tests were considered, and some new tests 
were developed at ETS. The present Kit presents 74 factor-referenced tests 
for 23 cognitive factors, that have been "established" in thfe literature and, 
at /least partially, substantiated empirically. All of these tests were 
developed at ETS in order to facilitate th^ir use by researchers .> For the 
noncognitive area, we are/at roughly the sam^e stage that th^ cognitive area 
•was in 20 years ago. We are proposing a f first Guide itt) noi^ognitive factors 
with the bulk of the refereftape material emanating from many researchers and 
only the beginning of reference scales, in line with the existing literature, 
developed at ETS. ^ 

Summary lists of the. 23 cognitive factors with the recommended marker 
tests, and the 28 temperament factors with the scales proposed for them are 
giv^ in th^ Xppendlx. > 
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Appendix I q ^ 

MASKER TESTS FOR 23 COGNITIVE FACTORS { 



> 



CF Closure, flexibility of' 

1. Hidden Figures Test 

2. Hidden Patterns Test 

3. Coijying.Test 



CS Closure, Speed of 

1. Gest^alt Completion Test 

2. - Concealed Words. Test 

3. Snowy Pictures ^ 

CV Closure^ Verbal 

1. ScraBibied Words 

2. Hidden Words 

3. Incomplete Words 

FA Fluency, Associational 

1. Controlled Associations 

2. 0pposit6s Test 

3. Figures pf Speech 

FE Fluency, Expressional • 

1. Making Sentences 

2. Arranging \ Words 

3 . Rewriting 

FF Fluency, Fig^ral 

1. Ornamentation Test 

2. Elaboration Test 
3,.^ Symbols Test if^. < 

FI> Fluency, Ideatloiritl 

. 1. Topics Test , 

2. Theme Test 

3. Thing Categories Test 



FW ^^^^ency. Word 



[est 



1. Word Endings Test 

2. Word Beginnings Test 

3. Word-- Beginnings, and Endings Test 



, Induction'* 

1. Letter Sets Tdst 

2. Locations^ Test 

3. Figure Classification 

;P Integrative Processes 

1. Calendar Test 

2. Following Directions 
3* Language Rules 



MA Memory, /Associative ' , 

1. Pictute^Number Test 
2\ Object-^Number Test 
• 3. First & Last Names Test 

MS Memory Span 

1. Auditory Ntjmber Span Test 

2. Visual Number Span Test 
3* Auditory Letter Span Test 

MV Memory, Vi^ua^ 

1. Shape Memory Test 

2. Building Memory 
3* Map Memory 

N Numbla . ^ 

rr^ Addition^ Test 
2. Division Test 

3» Subtraction & Multiplication Test 
4. Addition & Subtraction Correction 

P Perceptual Speed 

1. Finding A's Test 

2. Number Comparison Test 

3. Identical Pictures Test 

R6 Reasoning, General^ 

1. Arithmetic Aptitude Test 

2. Mathematics Aptitude Test 

3. Necessary Arithmetic Operations Test 
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RL Reasoiiiiig, Logical 

1. Nonsense Syllogisms Test 

2. piagracQm±ng Relationships 

3. r Infeirence Test 

4. Dec4.pharing Langtiages 

S Spatial Orientation 

^1. Card Rotations Test ' / 

2. Cube Comp^isons Test 

3. Spatial Aspects ^ 

SS Spatial Scanning 

1. Maze-Tr^a^ng Speed Test 

2. Choosing^' A Path 

3. Map Planning Test 

V' Verbal Comprehension J^r 

1. Vocabulary Test I, 

2. Vocabulary Test II 

3. Extended Range Vocabulary Test 

4. Advanced Vocabulary Test I 

5. Advanced Vocabulary Test II 

VZ Visualisation" 

1. Form Board Test 

2. Paper Folding Test • 

3. Surface Development Test 

« « 

XF Flexibility, Figural 

1. Toothpicks Test 

2. Planning Patterns 

3. Storage Test \ 

XU Flexibility of Use 

1. Combining Objects 

2. Substitute Uses 

3. Making Groups 

4. Different Uses 
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AppBndix'II o 
MARKER SCALES FOR ^§ TEMPERAMENT FACTORS 



Ac Geueral Activity ' , 

1. Moves rapidly, quick in physical performance vs. slow . || 

2. Busy, active ±ri projects or <^nsocjLal affairs vs. uninvolve^*^ feels 

overburdened d - " | ] 

3. ' Vigorous, healthy vs. tired, lacks energy 

Ag Agreeableness ^ . . ' - i 

1. Coopfer^tive, supportive, forgiving vs. irritated by people, v^ngeftxl ^ 

2. Adaptable, tends to' agree, sub^fisive vs. negativistic^ dominfeeriiig - 

3. Trustful, confides in pepple^^^^. suspicfbufi, keeps distance ~ | . ^ 
'"4. Friendly, likeable, outgVirig vs. aloof, unpleasant, ^wlthdraim^^ 

If |t| 

Al Alertness* i ^ ^' 1' m\- 

' . - ' " f \ ' ^ 
'Alertness to immediate surroundings, attentive vs. unaware^ e^g^ossed, 

deep in thoughtu absentminded 1 ^ • 

Au Autistic Tendency / * ' .J 

1. Daydreams or has practical thoughts . 
' 2. Bothered by daydreams or autistic thinking vs. enjoys thD^^'t3yL^ 

Ca-^^^lmness vs. Anxiety { 

1. Relaxed, stable, at ease vs. anxious, worried (about self),yedgy, uneasy 
.2. Relaxed, adjusted, realistic thoughts vs. anxiety and wprry that leads 

to autistic thinking ! 
3., Physically relaxed vs. fidgets, has nervous habits, twit:ches, makes 

restless movements. , / 

Co Concent rati j^on . , , * 

1. Concentration on study or reading, restraint leading to maintenance of 
attention vs. mind wanders, bpred, forgets names • ^ 

/ . / ' * ^ . • r^; .■ 

De Dependability ' v 

1. Conscient&oup, scrupulous vs. careless about doing 'what is tight 

2. Dependable^ punctual,. k§eps promises vs. careless, about .^bmises and 

details " ' 

3^. Self-sentiment coptrol, control of own feelings vs. action?" and thoughts 
are swayed by emotions 

Do Dominance ' ' 

1.. Takes charge socially, wants power vs. submissive, wiliing to serve 

2. -Egoistic, pushes own ideas vs. respects others* ideas, self-effacing 

3. Rights-donscious, complaining vs. tolerant. . 



Em Emotional Maturity 

1. Patient, adjusts to frustration vs. verbally aggressive, de m a n ding 

2. Modest, shi^ f attention, OTutwardly, directed vs. self-centered | seeks 

' attention, .egotistical 
• 3. Satisfied, cooperates with authority vs. ^asserts It^ependence from 
authority, stubborn , , '* • ^ 

0^, Tolerant of physical, nonhtaoan, or situational annoyances vs. irjdtated 

by mishaps and frustrating circumstances 
5. Tolerates the simper feet ions in -things vs. feels hostility toward things 
1^^^ fair to work ^ * ^ ( 

Es Emotiofial Stability ^ ' * , ' l. 

1. Emotionally stable, toierant, stolid vs. emotionally sensitive, irritable 

2. Optimistic, faces problems vs. worrying, dwells on problems, escapist 

3. Feels health^ vs. hypctcfiiondriacal ; 

Gs Gregariousness ^ 
1; I^ikes to be alone ^ 

2. Likes working or socializing witl^ people vs. likes work a2,one or isdLated 

'■a. . ^" ' 

In Individualism' • - 

1. Desires to be different, individualistic, free vs. needs appl1co3/al of 

■ if others, conforms, accepts tihe social order, agrees witTi group, likes 
">aff illation, complies 

2. lias unusual ideas, unconventional, idealistic, reflecti^jg vs. has 
^ majority o^oLniqns, tends to have* same feelings as others 

Me Meticulousness . ' 1^ 

1. Meticulous, orderly, neat, careful, particular about personal -effects " 

2. Not messy, careless, or impulsive 

3. Conscientious, careful, exactlaig, tidy, orderly 

'1 \ ■ : ' ^ , 

Mo l^rality ^ * 

1. ' Law-abiding, obedient, well-mannered, patriotic vs. free, progressive, 

liberal * \ a 

2. Moral, knows right from^wfong, resista temptation vs. ^pleasure seeking 

3. Helpful, fair to peopl^^ 

Na "^eed for Achievement ^ \^ ' 

1. Likes success in competition, ^ikes getting ahead vs. dislikes 

competition 

2. Strives for accomplishment, wants to produce something great^ 

Ob -Objectivity vs. Paranoid Tendency 

1. Objectivity antf fairness attributed to others vs. paranoid delusions 

2. Credit is given by others vs. blame by others is unfair 
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Om Open-mind edness vs. Dogmatism ♦ ' . . 

, 1. Beliaves many different philosophies (religious or political views) ^an 
be treasonable vs. 'rigid belief in pne^ philosophy, no tolerance of \ 
compromise \ . 

2: Respect Eor'^nd interest in the reUgiQus ' an^ political philosophies of * 
^ ^ ^ cytjiier ^people vs;. strong belief in the righttlesi's or wrongness of 
principles ' ^ ^ ' ' . 

' 3. Innovative, ready for n^Seas, -flexible, fpr<es^^ W. highly 



consexvativiB, conventioi^iir, and unchangeaible ideas 

Pe Persistence' 14 
-1. Persistent, persevering, determined vs. quitting,/ fickle, needs change. 



\ gets discouraged 

2. (The reverse of ) play before work 

Pd Poise vs. S.elf -Consciousness v- 

1. Enjoys group 'attention, exhibitionistic, ppised vs. dislikes being li^ 

front of people ^ 

2. ^Enjoys performing in public, feels pride in speaking to a group v^ 

dislikes performing in public 

3. Withdrawn, fears public speaking and social responsibilities 

*■ 

Rt Restraint vs. 'Rhathymia 

1. Planning vs. acting without thought, impulsive ^ " 

2. Serious, responsible vs^. likely, carefree, irresponsible, no thought 

of the future 

3. Enjoys^ stable pursuits vs. wants excitement, change, w^ldness 

Sc Sel:^-Conf idence . . / , 

1. Feels confident physically, personally, and career-wise vs. needs 

encouragement, feels inferior, afraid of failure 

2. Claims to have abilities,, skills, and good experiences vs. xilaims 

handicaps, ineptitude, and unfavorable experiences ^ 

3. Perceives others as having been positive toward him vs. negative 

Se Sensitive Attitude " • 

1. Warm, soft, cooperative, kiW, considerate vs. hard, stem, bossy 

2. Emotionally sensitive, ' empatltietic, delicate,* quiet ^ > 

3. Interest in people's welfare, religion 

4. Interested .in people's welfare ^ helpful 

5. Selfish, uncharitable . ^ 

6. Motivation to do good or to help pedple 
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So Sociabxlity 

1. Glib talker, has superficial social know-how vs, aloof, doesn't know 

or care what should be said 

2. Hardened socially, confidfent in social contacts va^ shy, socially 

insecure ' • " ^ ' * 

3. Competent; socially, social organizer, enjoys attention vs. withdrawn, 

fears "public speaking and social responsibilities' 

Ss Self-Sufficiency . ^ / 

1, Self-sufficient, likes tb be alone in stress, in planning,. In facing^ 

problems, makes own plans,, dislikes "being serjred, self reliant, 
decisive vs. dependent, needs help from others, group dependent 

2. Emotional independence vs. needs iove, friends, succorance, and 

protection ^ ^ 

*** ^ 
Su Surgency " • B . ? 

1. Exuberant, enthusiastic, cheerful vs. repressed, reserved,,' inhibited 
2» Talks without inhibition, expressive, frank 

Th Thoughtfulness - - ; . . 

y - • 

1. Likes to think, reflect, meditate vs. prevented from doing it by 

social or business activity , . 

2. Likes to think aboiit people or with people vs. enjoys the company 
^ of people without analyzing them 

3. Thinks about self vs. carefree about ^self 

4. Intellectual interests vs. active interests 

To Tolerance of Human Nature vs. Cynicism 

1. Naive, impunitive, believes. people are honest and fair vs. believes 

people lie and are unfair to gain an advantage ^ 

2. Believes people' are capable of good work vs. critical, fault finding 
3» Tolerant of human nature vsJ cynical aboutf human nature 

Wb Well-being vs. Depression f - , . ' 

1. Has feeling of well-being, euphoria vs. depressed, blue, lonely 

2. Hopeful, interested in life vs. fear and worry about doom or vague 

- dangers ^ _ " . .° 

3. Confident, can stand criticism vs. giiilt prone, feels worthless and ^ 

spurned, worries about self ^ 
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